1. Field of the Invention
The present invention relates to a video signal reproduction apparatus which is capable of reproducing bitstreams containing audio data, video data, sub-picture data, and additional information which are digitally encoded by an encoding technique (e.g., MPEG) and multiplexed on a pack-by-pack or packet-by-packet basis.
2. Description of the Related Art
Recent years have seen a rapid and wide spread of media capable of accommodating video signals, audio signals, and additional information (e.g., subtitle information) in a digitally encoded and multiplexed format. One example of such media is the DVD (digital video disk).
Reproduction apparatuses for such multiplexed digital signals are associated with the following problems, which may hinder the smooth and normal operation of the reproduction apparatus: input of non-continuous and inconstant bitstreams during a special reproduction mode (such as a forward skip or a backward skip, which may occur without continuous data input); generation of errors at the time of encoding; generation of errors due to damaged disks or noises present on a transmission path; and the like.
On the other hand, the reproducing of audio, video, and sub-picture data generally requires synchronization. If the bitstream has been encoded with a tendency toward underflowing, it is possible that, when skipping sub-picture data by reproduction units, reproduced data may not be complete, thereby hindering proper reproduction.
In the field of digital media, video signals are generally subjected to an encoding scheme that complies with the MPEG standards, while encoding schemes other than MPEG-compatible schemes may be adopted for audio signals. A special bit map data compression technique is adopted for the encoding of subtitles carried on DVDs. The multiplexing of encoded audio or video data is performed by a multiplexing method which complies with the MPEG system standards.
FIGS. 2A and 2B illustrate code sequences which are multiplexed on the basis of packets 102. FIG. 2A illustrates the structure of a pack 101 as a basic unit of multiplexing. At the beginning of a packet 102, a packet header 107B is added which includes a synchronization signal (hereinafter referred to as a xe2x80x9cpacket start code prefixxe2x80x9d) 103 indicating the top or beginning of the packet, a packet identifier (stream_id) 104 for distinguishing video packets from audio packets, packet length information 105, and video/audio synchronization reproduction information 106. Although FIG. 2A illustrates an example where the pack 101 includes one packet 102, the pack 101 may alternatively include a plurality of packets 102.
FIG. 2B illustrates an exemplary structure of a multiplexed code sequence 108 and sub-picture encoded data 109. The multiplexed code sequence 108 contains audio packs AP, video packs VP, sub-picture packs SP, and navigation information encoded data packs NP. The audio signal, the video signal, the sub-picture signal, and the navigation information are respectively digitally encoded by an encoder, and multiplexed by a multiplexer on a pack-by-pack basis.
The sub-picture encoded data 109 in FIG. 2B is shown as only including the sub-picture encoded data of the sub-picture packs SP extracted from the multiplexed code sequence 108 and combined together. An shown in FIG. 2B, each reproduction unit 110P and 110Q of the sub-picture encoded data 109 includes a unit header 110A, an encoded bit map data 110B, and display control information 110C corresponding to the bit map data 110B.
Conventionally, the sub-picture encoded data 109 is reproduced by analyzing the unit header 110A at the beginning of the reproduction unit 110P or 110Q, decoding the encoded bit map data 110B by using the information described in the unit header 110P or 110Q, performing display control functions (such as setting the output timing or adjusting color changes by using the display control information 110C for the decoded bit map data 110B), and blending the decoded bit map data 110B with the video reproduction signal for output. The transition from the reproduction unit 110P to the reproduction unit 110Q is achieved by moving the reproduction position to the beginning of the reproduction unit 110Q in accordance with a reproduction unit length described in the unit header 110A, and consecutively performing reproduction.
Conventionally, the beginning of the reproduction unit 110P of any viable sub-picture encoded data 109 (that is free of errors and the like) may be detected by relying on the reproduction unit length described in the unit header 110A; and the reproduction unit length is used for reproducing the sub-picture encoded data 109. The illustrated sub-picture encoded data 109 does not itself include a synchronization pattern. Rather, the beginning of the reproduction unit 110P of the sub-picture encoded data 109 starts with the unit header 110A. The unit header 110A cannot be detected by encoded data matching based on a synchronization pattern, as would be performed for video or audio data.
Even if the reproduction unit length described in the unit header 110A includes an error for some reason, it may still be possible to continue reproducing some or all of the reproduction unit 110P that is associated with the wrong reproduction unit length. However, the erroneous reproduction unit length makes it difficult to properly move the reproduction position to the beginning of the next reproduction unit 110Q. Therefore, the unit header 110A in the reproduction unit 110Q cannot be detected. Since it is difficult to decode the reproduction unit 110Q, a sub-picture decoder used for decoding the sub-picture encoded data 109 may hang up. Thus, if an error is present in the unit header 110A, it becomes difficult to smoothly reproduce the sub-picture encoded data 109.
FIGS. 3A and 3B illustrate a data transfer scheme in a special reproduction mode. In FIG. 3A, sub-picture packs SPU1_1, SPU1_2, and SPU1_3 correspond to the reproduction unit 110Q of the sub-picture encoded data 109: and sub-picture packs SPU2_1, SPU2_2, and SPU2_3 correspond to the reproduction unit 110P of the sub-picture encoded data 109. In the continuous multiplexed code sequence 108A shown in FIGS. 3A and 3B, if a pack within a period DT1 and a pack within a period DT2 are reproduced in an intermittent manner in a special reproduction mode, the reproduction unit 110P for the sub-picture data will not be completed, and the sub-picture packs SPU1_1 and SPU1_2 and the sub-picture pack SPU2_1 in the next reproduction unit 110Q will be combined into, and processed as, decoded date 111 as shown in FIG. 3B.
The sub-picture encoded data 109 is reproduced in such a manner that any data present in the uncompleted sub-picture reproduction unit 110P is reproduced based on the reproduction unit length described in the unit header 110A (which is included at the beginning of the reproduction unit 110P).
FIG. 4 illustrates the reproduction of the sub-picture encoded data 109 during intermittent reproduction. A decoding operation is performed so as to first reproduce the reproduction unit 110P by extracting the reproduction unit length described in the unit header 110A, reproducing the reproduction unit 110P, and then reproducing the next reproduction unit 110Q by commencing a reproduction operation from a position which is distant from the reproduction unit 110P by the specified reproduction unit length 113.
However, as shown In FIG. 4, the reproduction unit length 113 may not be equal to the actual data length of a reproduction unit 113A which is stored in a sub-picture bit buffer 13. Since the actual data length of a reproduction unit 113A which is stored in a sub-picture bit buffer 13 is smaller than the reproduction unit length 113, the decoding operation could be continued past the data portion in the reproduction unit 110P, and over to the data portion in the reproduction unit 110Q. However, since any data in the reproduction unit 110Q is not the data in the reproduction unit 110P, it is impossible to continue a normal reproduction operation. In fact, depending on the content of the reproduction unit 110Q, which has been reproduced as if it were the reproduction unit 110P, abnormal reproduction control may be performed so as to result in outputting an inappropriate reproduction signal as reproduction data; or the reproduction operation itself may even come to a halt.
If the specific content of the data in the reproduction unit 110P happens to be decodable, then the sub-picture decoder will begin reproducing the next reproduction unit 110Q. However, since the sub-picture decoder will rely on the wrong reproduction unit length 113 from the reproduction unit 110P, the sub-picture decoder will commence reproduction from midway, rather than the exact beginning, of the reproduction unit 110Q. As a result, the reproduction of the sub-picture data in the reproduction unit 110Q will similarly result in an abnormal operation, again causing malfunctioning of the sub-picture decoder. This would result in outputting an inappropriate reproduction signal as reproduction data, or the reproduction operation itself might even come to a halt. In the case where the reproduction operation stops, the reproduction apparatus must be reset in order to resume a decoding operation.
Usually, detection of a hang-up is time-consuming, and the particular input sub-picture data which was being reproduced is usually lost, thereby hindering smooth reproduction.
The abnormal operation of the sub-picture decoder due to abnormality of sub-picture data may occur not only at the time of intermittent data inputting during a special reproduction mode, but also in a case where the reproduction unit length does not match the length of the actual reproduction data due to an error associated with encoding or an error that is present on a transmission path.
As described above, the conventional techniques have a problem in that a sub-picture decoder has difficulties in performing continuous reproduction of sub-picture data during a special reproduction mode or in the presence of errors associated with encoding, errors due to damaged disks, or errors due to noise on a transmission path.
In one aspect of the present invention, there is provided a video signal reproduction apparatus for reproducing a multiplexed encoded stream including audio data, video data, sub-picture data, and additional information digitally encoded and multiplexed on a packet-by-packet basis, wherein the multiplexed encoded stream includes a plurality of packs, each of the plurality of packs including at least one packet, the at least one packet including a packet header and packet encoded data; and wherein the packet encoded data includes at least one of packet audio encoded data, packet video encoded data, packet sub-picture encoded data, and packet navigation encoded data; wherein the packet audio encoded data forms sub-picture encoded data, the sub-picture encoded data including a first reproduction unit and a second reproduction unit; the first reproduction unit including a first unit header, first encoded bit map data, and first display control information; and the second reproduction unit including a second unit header, second encoded bit map data, and second display control information, wherein the video signal reproduction apparatus includes: an input processing section for adding a next-sub-picture reproduction unit pointer to the first reproduction unit; and a sub-picture decoder for decoding the sub-picture encoded data based on the next-sub-picture reproduction unit pointer added to the first reproduction unit.
In one embodiment of the invention, the input processing section adds the next-sub-picture reproduction unit pointer to the beginning of the first reproduction unit.
In another embodiment of the invention, the input processing section adds the next-sub-picture reproduction unit pointer immediately after the first unit header.
In still another embodiment of the invention, the input processing section includes: a stream separation section for detecting the beginning of the first reproduction unit; and a stream transfer control section for adding dummy data to the first reproduction unit.
In still another embodiment of the invention, the stream transfer control section includes: a dummy data insertion section for adding the dummy data to the first reproduction unit; a data transfer section for transferring the first reproduction unit, to which the dummy data has been added; and an input reproduction unit length measuring section for counting the number of data in the first reproduction unit which has been transferred by the data transfer section, wherein the data transfer section overwrites the next-sub-picture reproduction unit pointer to replace the dummy data based on the number of data as counted by the input reproduction unit length measuring section.
In still another embodiment of the invention, the video signal reproduction apparatus further includes a sub-picture buffer for storing the first reproduction unit, to which the next-sub-picture reproduction unit pointer has been added by the input processing section, wherein the sub-picture decoder decodes the sub-picture encoded data stored in the sub-picture buffer.
In another aspect of the invention, there is provided a video signal reproduction method for reproducing a bitstream including audio data, video data, sub-picture data, and additional information digitally encoded and multiplexed on a packet-by-packet basis, wherein the multiplexed encoded stream includes a plurality of packs, each of the plurality of packs including at least one packet, the at least one packet including a packet header and packet encoded data; and wherein the packet encoded data includes at least one of packet audio encoded data, packet video encoded data, packet sub-picture encoded data, and packet navigation encoded data; wherein the packet audio encoded data forms sub-picture encoded data, the sub-picture encoded data including a first reproduction unit and a second reproduction unit down stream from the first reproduction unit; the first reproduction unit including a first unit header, first encoded bit map data, and first display control information; and the second reproduction unit including a second unit header, second encoded bit map data, and second display control information, wherein the method includes: a first step of adding a next-sub-picture reproduction unit pointer to the first reproduction unit; and a second step of decoding the sub-picture encoded data by using the next-sub-picture reproduction unit pointer.
In one embodiment of the invention, the first step includes adding the next-sub-picture reproduction unit pointer to the beginning of the first reproduction unit.
In another embodiment of the invention, the first step includes adding the next-sub-picture reproduction unit pointer immediately after the first unit header.
In still another embodiment of the invention, the first step includes adding the next-sub-picture reproduction unit pointer to the first reproduction unit during a special reproduction mode involving intermittent data input.
In still another embodiment of the invention, the first unit header contains a unit header length representing a reproduction unit length which is added when the packet sub-picture encoded data is encoded, and the second step includes using the next-sub-picture reproduction unit pointer with a priority over the unit header length.
In still another embodiment of the invention, the second step includes: determining an input data error in a case where the next-sub-picture reproduction unit pointer does not coincide with the unit header length, and decoding the second reproduction unit by detecting the beginning of the second reproduction unit by using the next-sub-picture reproduction unit pointer.
In still another embodiment of the invention, the second step includes skipping to the second reproduction unit based on the next-sub-picture reproduction unit pointer.
In still another embodiment of the invention, the second step includes, in a case where the first display control information includes valid first display control information, decoding the first reproduction unit based on the valid first display control information.
In accordance with one aspect of the invention, a sub-picture decoder can utilize the actual reproduction unit length of input sub-picture encoded data, instead of a specified reproduction unit length of sub-picture data which may have been destroyed by errors of the like. As a result, the sub-picture decoder can always know the accurate unit header position which marks the beginning of a reproduction unit of sub-picture encoded data.
In accordance with another aspect of the invention, the reliability of sub-picture encoded data reproduction in an inconstant state can be improved, e.g., during a special reproduction mode or in the presence of errors.
Thus, the invention described herein makes possible the advantages of (1) providing a video signal reproduction apparatus and a reproduction method which are capable of performing proper reproduction, without hanging up, in a special reproduction mode during which intermittent data may be input, or in the presence of errors generated at the time of encoding, errors due to damaged disks, and/or errors generated due to noise on a transmission path; and (2) providing a video signal reproduction apparatus and a reproduction method which are capable of decoding at least a reproducible portion of sub-picture data in a special reproduction mode during which intermittent data may be input, or in the presence of errors generated at the time of encoding, errors due to damaged disks, and/or errors generated due to noise on a transmission path.
These and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.